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Introduction 


Science learning is a lifelong endeavor (National Research 
Council, 2009). It takes place not only in school, but also out of 
school (Resnick, 1987). Eshach (2007) distinguished out-of-school 
science learning into non-formal and informal categories: non- 
formal science learning occurs in a planned but highly adaptable 
manner in science centers, science museums, and other public sci- 
entific institutions beyond the spheres of formal schools; whereas 
informal science learning occurs in day-to-day routines that come 
about spontaneously around the home, street, playground, and 
SO on. Science centers and science museumMs serve as important 
infrastructures to enable out-of-school science learning (Dierking, 
1997; Falk, 2001; Falk & Dierking, 2000), especially in enhancing an 
individual's affective learning outcomes toward science (Rennie 
& McClafferty, 1996; Rennie & Williams, 2002). The advent of the 
Internet has created new opportunities for the development of 
activities relating to learning at or with science centers and sci- 
ence museums. The most remarkable advantage of integrating 
science centers and science museums with the Internet is the 
diminution in the barrier of physically visiting science centers 
and science museums; web-based environments enable a wide 
range of Internet users, including school students, researchers, 
and the general public, to virtually surf science centers and science 
museums around the world without geographic and time limits 
(Corredor, 2006; Jones-Garmil, 1997). 
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Abstract. A short look on any science 
center or science museum website reveals 
that significant amounts of online educa- 
tional resources have been developed in 
recent years. However, how can the non- 
formal online learning resources of science 
centers/museums support learning activi- 
ties inside schools? This study leverages the 
educational pathway of energy resources, 
designed by the European Open Science 
Resources project and the Deutsches 
Museum, to develop in-school learning 
activities. This research explores the impact 
of different instructional approaches incor- 
porating the educational pathway, includ- 
ing the Self-Guided Educational Pathway 
(SGEP) and Teacher-Guided Educational 
Pathway (TGEP), on Taiwanese high-school 
students’ science learning outcomes. The 
results indicate that the TGEP approach 
provides students significantly higher 
knowledge gains than the SGEP approach. 
Moreover, the TGEP approach significantly 
maintained students’ positive attitudes 
toward science learning, museum learning, 
and online museum learning than did the 
SGEP approach. The results are discussed 
in terms of both pedagogical designs and 
the social culture of Eastern Asia. 
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A short visit to a website of a science center or science museum reveals considerable amounts of 
online resources aiming to support learning activities both on-site and off-site. However, what are the 
resources that science museums and science centers can make available through the Internet? And, 
how can they support learning activities inside and outside school? So far, little research has addressed 
these issues. These and other questions are presently being addressed within the European Open Science 
Resources (OSR) project by a consortium of science centers, science museums, universities, educational 
bodies, and researchers in science education from the European Union, Taiwan, and the United States 
(see http://www.openscienceresources.eu). 

The OSR project aims to harvest and organize the educational resources from different contribut- 
ing institutions into a new educational framework supporting in-school and out-of-school learning 
using the OSR web portal (see http://www.osrportal.eu). The OSR portal online exhibits the educational 
resources managed by 7 science centers and science museums in Europe. Online educational resources 
are presented in English or another out of 8 different main European languages. The immediate aim is 
to make educational resources on natural science more readily accessible and findable. The framework 
itself introduces a set of standardized metadata providing clear descriptions of the intended context of 
usage of the educational resource and its objectives. In addition to the standardized description pro- 
vided by the creator of the resource (i.e. an expert in the field), the users of the portal themselves have 
the liberty to classify the resources with additional free keyword by means of social tags. The liberty 
of tagging resources can improve the efficiency to find educational resources and provide additional 
contexts to the resources and their usage. 

In order to guide visitors in physical or virtual visits, the OSR portal provides tools to assemble 
learning scenarios by combining patterns of the educational resources into educational pathways. An 
educational pathway can be a physical or a virtual activity using the educational resources of the science 
centers and science museums on a given theme. A short summary (including title, classification, short 
description, and education objectives), metadata, and social tags (users must log in to see the tags) are 
presented on the entry page of each educational pathway. Each educational pathway also contains a 
set of multimedia files, including pictures, documents, hyperlinks, animations, and simulations. There 
are 220 educational pathways exhibited in the OSR repository to date (05/28/2012). These pathways are 
designed both for the usage in the in-school context (e.g., by school teachers for classroom activities) 
and out-of-school context (e.g., by general visitors who would like to learn on the web more about the 
content at the science center or science museum). 

As acontent provider for the OSR project, the Deutsches Museum is involved in the development of 
resources including collaborative pathways on a single topic with museum partners from other European 
countries, such as the Cité de la Science in Paris and the Museo della Scienza e della Tecnologia“Leonardo 
da Vinci” in Milan. This collaborative work of aggregating educational resources from various museum 
sites into ona common theme is one of the potential strengths of the OSR portal. The Deutsches Museum 
has designed a model pathway on the theme of “Energy is everywhere: Historical and contemporary 
power generation” to be used in the in-school context. The pathway includes a number of resources 
relating to the collections and exhibitions of the Deutsches Museum, and is complemented through a 
number of web-based resources on the theme of energy conversion and power generation completing 
the pathway to a possible series of units for the classroom. Due to being, in principle, disconnected from 
a physical visit to the museum, the usage of the educational pathway however faces immediate chal- 
lenges: How can we design instructional approaches appropriately to bridge in-school and out-of-school 
learning? What is the impact of integrating the content of out-of-school learning into school curricula 
on students’ cognitive and affective learning outcomes? Only a test case, which is the main focus of the 
current study that was run outside of the country where the Deutsches Museum is located, can show 
whether the intended scenario of usage is practical outside the original cultural scope. 

Therefore, the Science Education Center of National Taiwan Normal University and the Deutsches 
Museum collaboratively conducted this pilot study to explore the impact of different instructional 
approaches incorporating the educational pathway on Taiwan high-school students’ science learning 
outcomes, in terms of both cognitive and affective dimensions. The hope is that the results of this pilot 
study can inform educational practitioners how to use non-formal learning resources of science centers/ 
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museums to bridge in-school and out-of-school science learning. The OSR portal includes a function to 
adapt existing resource patterns, such that they can more easily be modified and included into in-school 
activities based on local curricula. 


Methodology of Research 
Educational Pathway of Energy Resources 


The educational pathway used in this study is called “Energy is everywhere: Historical and con- 
temporary power generation” (see http://www.osrportal.eu/en/node/94960). This pathway is a virtual 
walk through selected collections at the Deutsches Museum in Munich and additional resources on 
the world-wide-web. The content of this educational pathway is presented in English only. As shown 
in Figure 1, the educational pathway is divided into 3 units, including: “What is energy? A basic review 
of physics terminology,’“From the ox treadmill to photovoltaics: Usage of energy then and now,’ and 
“Enough energy for all? Assessing the potential of future technologies.” Each unit explicitly states learning 


Wu 


objectives for visitors, such as “define work, energy and power in words and link between them,’“relate 
different engineering methods to power generation,’“identify the key characteristics of the methods,’ 
and “understand some of the limitations of power generation.” Brief explanations of physics concepts 
related to energy resources, such as work, power, and energy conservation, are given with the online 
lectures at MIT OpenCourseWare (e.g., Prof. Walter Lewin Physics | lectures). Different types of energy 
resources, such as wind energy, nuclear energy, and solar energy, are introduced with the exhibits of 
the Deutsches Museum, news in daily life (e.g., the project of International Thermonuclear Experimen- 
tal Reactor), and scientific simulations (e.g., nuclear fusion simulation of the University of Colorado). 
Assignments are located in each unit to stimulate visitors to evaluate their learning progress, such as 
“Discuss good locations for a wind farm. Where is the world’s most powerful farm presently located?” 
The goal for visitors is not only to sharpen their understanding of energy resources, but also to assess 
which technologies might meet the ever growing demand for energy. The educational pathway of 
energy resources can be seen as an extension of the unit of “minerals, energy, and life” in the Taiwan 
high school earth science curriculum. 


What is energy? Usage of energy then & now 
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Figure 1: Three units of the educational pathway. (a) Unit 1; (b) Unit 2; (c) Unit 3. 


Learning Conditions 


Two different instructional approaches, namely the Self-Guided Educational Pathway (SGEP) and 
Teacher-Guided Educational Pathway (TGEP) approaches were designed in this study. A machine trans- 
lation website (i.e., http://translate.google.com/) was provided for both SGEP and TGEP approaches to 
help students overcome the language barrier. The SGEP approach emphasized autonomous learning; 
students were asked to accomplish the assignments allocated in the educational pathway by studying 
the educational pathway at their own paces. In contrast, the TGEP approach was the mixture of both 
constructivist-oriented and reproduction-oriented pedagogies (see Chang & Chang, 2010; Chang, Hsiao, 
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& Barufaldi, 2006; Chang et al., 2010). The high school earth science teacher involved in the TGEP ap- 
proach explicitly sets the themes for students to inquire as “What are renewable energy technologies,’ 
“Which technologies can meet the ever growing demand for energy,’ and “Which risks do the various 
methods bear?” As shown in Figure 2 (a), the teacher gave students an overview about the educational 
pathway, explained the physics concepts presented the educational pathway to students, and briefly led 
students to surf the collections demonstrated on the educational pathway. Students were divided into 
sub-groups and given time to construct group arguments regarding the aforementioned three themes 
for inquiry by surfing the educational pathway and discussions with peers. Finally, as shown in Figure 2 
(b), the teacher engaged each sub-group to present and articulate their arguments in an open discussion, 
compared and challenge students’ arguments, and made a conclusion from the open discussion. 


Lay 





Figure 2: Theinstructional condition of TGEP group. (a) The teacher led students to surf the selected 
collections demonstrated on the educational pathway; (b) The teacher engaged students 
in the open discussion on the modern energy issues. 


Participants and Research Design 


In this study, the sampling involved high school students because one of the most relevant in-school 
learning units to the educational pathway of energy resources is located in the high school earth science 
curriculum (e.g., the unit of “minerals, energy, and life”). Two public male high schools, school J and A, 
in Taipei were selected. During the month of April, 2011, a total of 96 male students from school J and 
A participated in this study. Students were assigned to the SGEP or TGEP condition by school unit; the 
students from school J (n = 69) received the SGEP condition, and the students from school A (n = 27) 
received the TGEP approach. The students’ mean age was 16 (ranged from 15 to 17, SD = 0.6). 

A pre-test/post-test comparison-group quasi-experimental design was adopted (Campbell & 
Stanley, 1966) to investigate the impact of SGEP and TGEP on students’ learning outcomes regarding 
knowledge of energy resources and attitudes toward science learning, museum learning, and online 
museum learning. As shown in Figure 3, the data collection consisted of four phases: (1) evaluating 
students’ English capability by using English Proficiency Test (EPT); (2) evaluating students’ prior knowl- 
edge of energy resources and pre-intervention attitudes toward science learning, museum learning, 
and online museum learning utilizing Energy Resource Knowledge Test (ERKT), Attitude toward Science 
Learning Inventory (ASLI), Attitude toward Museum Learning Inventory (AMLI), and Attitude toward 
Online Museum Learning Inventory (AOMLI); (3) implementing one-hour SGEP and TGEP interventions 
to the two groups respectively; (4) evaluating students’ knowledge gains and post-instruction attitudes 
toward science learning, museum learning, and online museum learning utilizing ERKT, ASLI, AMLI, and 
AOMLI. 
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Figure 3: Research design of the current pilot study. 


Measurements 
Evaluating students’ English capability 


All the information presented in the educational pathway was in English only, and the students 
were not native English speakers; thus, students’ learning outcomes might be affected by their English 
capability. Therefore, the EPT was developed to determine students’ English capability prior to the in- 
tervention. EPT was consisted of 20 multiple-choice items selected from TOEFL, and its feasibility was 
checked by two high school teachers and one postdoctoral researcher. EPT was administrated to the 
students before the intervention. The Kuder-Richardson Formula 20 (KR,,) was used to determine the 
internal consistency of EPT. The estimated KR,, coefficient was 0.77 for EPT. 


Evaluating Students’ Learning Outcomes 


To measure students’ learning outcomes in terms of both cognitive and affective dimensions, 
the ERKT, ASLI, AMLI, and AOMLI were developed in this study. ERKT consisted of 20 multiple-choice 
items. The items of ERKT put an emphasis on remembering and grasping ideas or concepts relevant 
to the content of the educational pathway. Two high-school teachers and one professor ascertained 
the content validity of ERKT. Based on Chang and Cheng's Inventory of Self-Confidence and Interest 
in Science (2008), ASLI, AMLI, and AOMLI were developed to assess students’ attitudes toward science 
learning, museum learning, and online museum learning, respectively. Each instrument consisted of 
six items, and its content validity was ascertained by the same panel of specialists. All the instruments 
were administrated to the two groups at the beginning and end of the intervention as the pre-post 
test format. The Cronbach's alpha internal consistency for pre-ASLI, pre-AMLI, and pre-AOMLI estimates 
were: 0.94, 0.95, and 0.97, respectively. For post-ASLI, post-AMLI, and post-AOMLI, the Cronbach’s alpha 
internal consistency estimates were: 0.99, 0.97, and 0.97, respectively. The KR,, internal consistency esti- 
mates were 0.34 for pre-ERKT, and 0.65 for post-ERKT. Some may question the reliability of ERKT as the 
KR, value for pre-ERKT is somewhat low; however, as pointed by Yu (2001), a low internal consistency 
estimate of a pre-test may result from random guessing when subjects have not been exposed to the 
treatment. Judging the reliability of the instrument based on the pre-test scores is premature. Since 
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ERKT contains novel concepts for students to learn and the KR,, value of post-ERKT is acceptable, the 
data collected from ERKT is judged sufficient for the following analyses. 


Data Analysis 


The major variable was the format of instruction, and the dependent variables were measures of 
student learning outcomes. A multivariate analysis of covariance (MANCOVA) was conducted on stu- 
dents’ post-test scores to examine whether there were differences in students’ knowledge gains and 
post-intervention attitudes toward science learning, museum learning, and online museum learning 
between groups, controlling for English capacity, prior knowledge, and pre-intervention attitudes. 
Before investigating the impact of different conditions on students’ learning outcomes, it is important 
to examine the equivalence between SGEP and TGEP groups because students were assigned to the 
conditions by school unit. The students from school J and the students from school A were deemed 
similar because: (1) school J and A are located very close to each other within the same geographic area 
of Taipei; and (2) these two schools’ percentile rankings on the Basic Competence Test (a test which is 
currently used as a national entrance exam to screen students’ applications for high schools in Taiwan) 
are the same. Moreover, in order to gain the inferential power of the statistical hypothesis testing in 
explaining the impact of SGEP and TGEP on students’ learning outcomes, students’ English capability, 
prior knowledge, and pre-intervention attitudes toward science learning, museum learning, and online 
museum learning were used as covariates to compensate for the potential initial differences between 
groups. The homogeneity-of-regression and homoscedasticity assumptions for MANCOVA were also 
examined. However, the two schools may have different cultures, such as differing social rules and 
norms, and these potential dissimilarities between schools may have an impact on students’ learning 
outcomes. The potential variation between school cultures is acknowledged as a limitation of this 
study. To meet contemporary calls for improvement in the interpretation and reporting of quantita- 
tive research in education (Rennie 1998; Thompson 1996), this study reported practical significance 
along with each statistical significance test by using the effect size index f calculating from the results 
of ANCOVA. Based on Cohen’s rough characterization (1988), values of 0.1, 0.25, and 0.4 were termed 
as small, medium, and large effect sizes, respectively. The statistical significance testing was conducted 
at the alpha = .05 significance level by using Statistical Package for Social Sciences (SPSS) version 15.0 
(IBM, Armonk, New York, USA). 


Results of Research 


Table 1 shows the results of the MANCOVA on the students’ post-test scores with their pre-inter- 
vention states as covariates. The MANCOVA conducted in this study met the homogeneity-of-regression 
assumption. All the interaction terms between students’ pre-intervention states and instructional ap- 
proaches did not attain the statistically significant level (all the p-values > 0.05). In other words, the ef- 
fects of students’ pre-intervention states on students’ post-test scores could be seen as the same across 
groups. The heteroskedasticity assumption for the MANCOVA was also met; the residuals of the MANCOVA 
models did not violate the assumption of normal distribution (all the p-values of Kolmogorov-Smirnov 
tests > 0.05). As shown in Table 1, the contrasts on students’ adjusted post-test scores between groups 
indicated that students in the TGEP group scored significantly higher than did students in the SGEP 
group on ERKT (p < 0.001, f= 0.41), ASLI (p < 0.001, f= 0.50), AMLI (p < 0.001. f= 0.47), and AOMLI (p < 
0.001, f= 0.51) with large effect sizes. It is important to note that students in the SGEP group changed 
their attitudes toward science learning, museum learning, and online museum learning from the positive 
dipole (mean > 3) to the negative dipole (mean < 3) after learning with the educational pathway. On the 
other hand, students in the TGEP group retained positive attitudes (mean > 3) toward science learning, 
museum learning, and online museum learning after they learned with the educational pathway. 
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Table 1. Descriptive data for students’ pre-tests, and results of the multivariate analysis of covari- 
ance (MANCOVA) on students’ post-test scores with pre-test scores as covariates. 


Pre-test scores Adjusted post-test score 
F (1, 89) Pp i 
SGEP TGEP SGEP TGEP 
Mean (SD) Mean (SD) Mean (SD) Mean (SD) 

EPT 16.1 (2.8) 14.9 (3.5) none none none none none 
ERKT 12.1 (2.3) 13.3 (2.0) 13.9 (0.3) 15.9 (0.5) 14.62* 0.000 0.41 
ASLI 4.1 (0.7) 4.4 (0.6) 3.0 (0.2) 4.3 (0.2) 21.81* 0.000 0.50 
AMLI 3.7 (0.9) 4.00 (0.7) 2.8 (0.1) 3.8 (0.2) 19.37* 0.000 0.47 
AOMLI 3.3 (1.0) 3.6 (0.8) 2.7 (0.1) 3.6 (0.2) 23.42* 0.000 0.51 


SGEP self-guided educational pathway, TGEP teacher-guided educational pathway, 
EPT English proficiency test, ERKT energy resource knowledge test, 

ASLI attitude toward science learning inventory, 

AMLI attitude toward museum learning inventory, 

AOMLI attitude toward online museum learning inventory. 

*p < 0.001. 


Discussion 


Although the SGEP and TGEP groups substantially had knowledge gains (12.09—+13.62 for SGEP 
group; 13.33—>16.52 for TGEP group), the knowledge gain of the TGEP group was significantly higher 
than the SGEP group with a large effect size. It may result from the TGEP approach provided students 
with systematic instructional content and an organized teaching sequence so that this helped students 
to grasp scientific facts and concepts more easily. It also made sense that a teacher-guided approach 
should lead to higher achievement than a self-guided approach since the former approach made clear 
to students what the objectives were. This approach helped to clarify which learning materials and 
information were most important. On the other hand, the SGEP approach left students on their own to 
handle large amounts of information associated with the energy resources so that the students were 
lost in the educational pathway and encountered difficulties in constructing meaningful knowledge. 

It is worth to note that the TGEP group had significantly better affective learning outcomes (i.e., 
attitudes toward science learning, museum learning, and online museum learning) than the SGEP 
group with large effect sizes. The findings echo to the series of the science learning environment re- 
search conducted by Chang (2005, 2006, 2008, 2010a, & 2010b) in Taiwan; Taiwanese students prefer 
a science learning environment where constructivist-oriented and reproduction-oriented pedagogies 
coexist. In the current study, the self-guided learning approach even turned students’ interests in real/ 
virtual museum visiting and science learning from the positive dipole to the negative dipole; whereas 
the teacher-guided learning approach still retained students’ attitudes in the same positive states. The 
differences in the affective learning outcomes between the TGEP and SGEP groups may arise from stu- 
dent culture background and characteristics. Perhaps due to the conformity views, which are deeply 
rooted in Confucianism as well as virtues in Chinese culture, Taiwanese students are generally quiet and 
passive learners, and like listening to teachers instead of enjoying self-learning (Chang & Mao, 1999). 
Students in the SGEP group may feel incapable of improving their learning performance without teach- 
ers structured guidance which they were used to; thus, the students expressed their unhappiness on 
the post-attitude measurements. The diversity of social culture should be considered as an important 
factor of instructional design while educators try to leverage on educational pathways to assist students 
in science learning. 

The contemporary philosophy in science education, particularly within the research community, 
emphasizes the constructivist-oriented pedagogy. Students are encouraged to be actively engaged 
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throughout the learning process with a high degree of self-regulation, and teachers adopt an internal 
control over the learning process of the classroom (Chang & Chang, 2010; Chang, Hsiao, & Barufaldi, 
2006; Chang et al., 2010). This pilot study does not oppose reforming science education in Taiwan with 
constructivist-oriented pedagogy although students appear to be not ready for self-regulated learn- 
ing. The more important issue is how to assist students to transform progressively their roles from 
passive to active learners without losing their interests in science. The TGEP approach in this study was 
a combination of constructivist-oriented and reproduction-oriented pedagogies such as combining 
whole-class presentation, interactive discussions among the teacher and students, cooperative learn- 
ing, and argumentation construction. The reproduction-oriented element emphasized direct guidance 
and clear explanations of important concepts to the students given by the teacher. The constructivist- 
oriented component focused on group activities, cooperative learning, and class discussions between 
the teacher and the students and among students. In this study, the TGEP approach was demonstrated 
to be a workable approach in the transition between constructivist-oriented and reproduction-oriented 
learning without frustrating students’ learning attitudes. The TGEP approach also successfully brought 
the content of non-formal learning (i.e., museum learning) into the formal school setting. The further 
issue emerging from the results of this pilot study is how to fade out gradually the teacher's scaffolding 
of the TGEP approach to make students become willing and skillful autonomous learners. 

The participating teacher's reflection and comments on the educational pathway used in the TGEP 
group were collected along with this study. The teacher has made some suggestions to improve educa- 
tional-pathway-integrated learning from the following three dimensions: (1) content of the educational 
pathway: it is difficult for students to connect what they learn in the educational pathway with their 
daily-life experiences; (2) the web-page design: the design of web-pages is a bit unattractive to students 
and may decrease students’ learning motivation; and (3) pedagogy: detailed instructional methods and 
suggestions should be added into web-pages for teachers. We are currently in the process of modifying 
our educational pathways based on these comments received and will try to explore issues further in 
terms of how to better bridge in-school and out-of-school science learning in future studies. 


Limitations 


It should be noted that the results and interpretations are limited by the characteristics of the stu- 
dents and the potential differences between the two schools. The sample consists of only male students, 
and the students’ English capacity was found to be good. Besides, the cultural difference between the 
two schools was not estimated, so it is unsure whether the school culture has a significant impact on 
students’ learning outcomes or not. Therefore, the generalization of the findings should consider the 
gender effect, English capacity, and school culture. Although the sample size involved in this study is 
rather small, the effect sizes derived from statistical significance testing provide many practical implica- 
tions for generalizing the results. In essence, a small sample size, like this study, is inherently more difficult 
to attain a statistically significant result than a large one is. Daniel (1998) even pointed out that a “SST 
[statistical significance testing] is largely a test of whether or not the sample is large” (p.26). It is quite 
common to observe a Statistical significance with a large sample, even if there is little practical effect 
actually. However, the statistical significances found in this study all attained the large effect size. It sug- 
gests that the findings, reported in this study, are more likely to be replicated in a larger sample size. 


Conclusion 


As the various areas of science are constantly advancing and the role of science in society can 
undergo important changes through the years, science centers and science museums with vocation 
to contribute to science education will necessarily be part of the development. The advances in con- 
temporary information and communication technologies make science learning resources much more 
accessible and make science teaching much more flexible than ever before (Chang, Chien, Chang, & 
Lin, in press; Chien, Chang, Yeh, & Chang, 2012). For science centers and science museums, the access 
to educational materials relating to the visit can be an additional way to engage with the public. On 
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the other hand, changes in appreciation of science have made it necessary to create new activities that 
make modern scientific research more transparent and encourage an early interest for contemporary 
science and technology. On-site and off-site activities (i.e. physical and virtual) of science centers and 
science museums should therefore be seen as complementary approaches to support the formal science 
learning. Further research for the efficient collaboration between science centers/science museums and 
schools is also needed. 
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